feat(provider): add aimlapi.com as a first-class LLM provider - #1

Description of the Change

Problem

PentAGI's README already advertises an aggregator tier next to its per-vendor providers, but reaching aimlapi.com meant hand-configuring the generic custom provider: one endpoint, one key, no model catalog, no per-agent defaults, no entry in the Settings UI, and no ctester validation report. grep -rIin "aimlapi" over the tree returned 0 matches before this PR.

Solution

Adds aimlapi as a first-class ProviderType, following the eight steps in CLAUDE.md → "Adding a New LLM Provider" and the shape of the last providers added (minimax, qwen, kimi). The gateway is OpenAI-compatible, so the provider is ~85 lines on top of the existing openaicompat base rather than a new client:

• backend/pkg/providers/aimlapi/ — provider, embedded config.yml (13 agent roles) and models.yml (6 models), attribution transport, tests
• provider.ProviderAIMLAPI + DefaultProviderNameAIMLAPI, registry entry, AllProviderTypes (this is what the REST layer's Valid() whitelist reads, so no separate change there)
• AIMLAPI_API_KEY / AIMLAPI_SERVER_URL / AIMLAPI_PROVIDER in config.go, .env.example, docker-compose.yml
• goose migration adding aimlapi to the PROVIDER_TYPE enum, plus database.ProviderTypeAimlapi
• GraphQL schema + regenerated generated.go / models_gen.go / types.ts, resolver cases
• frontend icon and provider label (aimlapi.com), installer wizard screen, ctester/ftester provider options
• README, backend/docs/config.md, backend/docs/database.md, .vscode/launch.json
• examples/tests/aimlapi-report.md — the ctester -agents all validation run every shipped provider config carries

Model roster. The 13 roles are spread over four model families, deliberately excluding the OpenAI/Anthropic/Google models the gateway also carries — this product runs offensive-security workloads and those vendors' guardrails false-positive on legitimate exploit-development content. That is the same reasoning recorded in examples/configs/openrouter.provider.yml, and the role→model mapping mirrors it so the two can be compared side by side.

deepseek/deepseek-v4-pro also ships in models.yml for selection in the UI. Every id was verified against both the id set and the aliases set of GET https://api.aimlapi.com/v1/models?include=all on 2026-09-03, and each is a canonical id, not an alias — an alias on this gateway can resolve to a different model than its name implies (deepseek/deepseek-chat is published as its own model and as an alias of deepseek-v4-flash). No model was removed from anything, so no deletion-on-catalog-absence judgement was made. A TestModelsAreCatalogIds unit test pins the config to the shipped catalog.

The enricher is the one place the mapping diverges from the OpenRouter config. Through this gateway minimax/minimax-m3 ignores both reasoning_effort:"none" and extra_body.thinking.type:"disabled" and still thinks, returning the chain inline in content wrapped in <think> tags rather than in reasoning_content. The enricher is the role that explicitly wants thinking off, so it stays on deepseek-v4-flash, which honours it. This is documented in config.yml; the same config passes the reasoning_off capability test on OpenRouter, so the difference is gateway-side.

Attribution. Requests to api.aimlapi.com carry HTTP-Referer and X-Title (pointing at PentAGI, per the OpenRouter convention — they identify the calling application, not the gateway) plus X-AIMLAPI-Source and X-AIMLAPI-Partner-ID. Implementation notes, all covered by tests:

• a RoundTripper wrapping a copy of the shared system.GetHTTPClient result, so the process-wide client is never mutated
• header map built fresh per provider, so no shared constant can be mutated across instances
• a caller's own header always wins; RoundTrip tags a clone rather than the request it was handed
• the host is re-checked per request, so a redirect cannot carry the headers off our origin
• attribution is scoped to aimlapi.com and its subdomains: pointing AIMLAPI_SERVER_URL at a LiteLLM proxy or a self-hosted gateway that fronts the API disables the headers instead of tagging another operator's traffic

The final commit, chore(aimlapi): fork-only placement — do not send upstream, contains nothing but ordering (registry, whitelist, wizard menu, .env.example, README/docs section order). It is separated so it can be dropped before this is offered upstream, where list ordering is the maintainer's call.

Type of Change

• 🚀 New feature (non-breaking change which adds functionality)
• 📚 Documentation update
• 🔧 Configuration change
• 🧪 Test update

Areas Affected

• Core Services (Frontend UI/Backend API)
• External Integrations (LLM/Search APIs)
• Documentation
• Infrastructure/DevOps

Testing and Verification

Test Configuration

PentAGI Version: develop (fork of ea66530)
Docker Version: n/a — built and run from source
Host OS: macOS (darwin/arm64), Go 1.27.1, Node 26.7.0, pnpm 11.20.0
LLM Provider: aimlapi (https://api.aimlapi.com/v1), real API key via AIMLAPI_API_KEY
Enabled Features: none (ctester runs standalone, no DB or stack required)

Test Steps

1. cd backend && go build ./... and go test ./... on a pristine checkout of the fork's default branch, to record a baseline.
2. cd frontend && pnpm install && pnpm typescript && pnpm lint && pnpm prettier && pnpm test, same baseline.
3. Apply the change; re-run all of the above.
4. go run github.com/99designs/gqlgen --config gqlgen/gqlgen.yml and pnpm run graphql:generate — the committed generated files are codegen output, not hand edits.
5. Real inference through the new code path: ctester -type aimlapi -agents all -groups all, which builds the provider through the registry and calls the live API for all 13 roles.
6. A two-turn tool loop against the live API through aimlapi.New(...).CallWithTools(...), with tools deliberately cleared on turn two.

Test Results

Backend — go build ./... exit 0.

The two remaining failures are pre-existing and unrelated (cannot read .env file: embedded provider not initialized — the installer's embedded assets are not generated in a plain source checkout). pkg/server/auth's TestUserCache_ConcurrentAccess failed on the baseline run and passed afterwards; it is a timing-sensitive concurrency test, not something this change touches. +2 ok = the new pkg/providers/aimlapi package plus that flake going green.

go vet clean on every package touched.

Frontend — pnpm typescript, pnpm lint and pnpm prettier all exit 0.

No test file fails after the change that did not fail before it. All of these failures are the same localStorage/Cannot read properties of undefined (reading 'getItem') environment issue in this sandbox, identical line-for-line to the baseline; three of them happened to pass on the second run.

Live provider validation — ctester -type aimlapi -agents all -groups all -workers 4, full output committed as examples/tests/aimlapi-report.md:

Agent          Model                       Reasoning   Success Rate     Avg Latency
simple         deepseek/deepseek-v4-flash  true        24/25 (96.00%)   1.717s
simple_json    deepseek/deepseek-v4-flash  false       7/7 (100.00%)    1.160s
primary_agent  z-ai/glm-5-turbo            true        23/24 (95.83%)   3.726s
assistant      z-ai/glm-5-turbo            true        22/24 (91.67%)   8.316s
generator      zhipu/glm-5.2               true        24/24 (100.00%)  3.893s
refiner        zhipu/glm-5.2               true        24/24 (100.00%)  3.708s
adviser        minimax/minimax-m3          true        23/24 (95.83%)   3.469s
reflector      deepseek/deepseek-v4-flash  true        25/25 (100.00%)  1.351s
searcher       deepseek/deepseek-v4-flash  true        24/25 (96.00%)   1.398s
enricher       deepseek/deepseek-v4-flash  true        24/25 (96.00%)   1.220s
coder          moonshot/kimi-k2-7-code     true        24/24 (100.00%)  4.091s
installer      moonshot/kimi-k2-7-code     true        24/24 (100.00%)  3.929s
pentester      deepseek/deepseek-v4-flash  true        24/24 (100.00%)  1.347s

Total: 292/299 (97.66%) successful tests
Overall average latency: 3.110s

For calibration against what is already in examples/tests/: openrouter-report.md 289/299 (96.66%), deepinfra-report.md 288/295 (97.63%), minimax-report.md 275/295 (93.22%). All four reasoning_off capability tests pass. The seven failures are model-behaviour variance (a model choosing a different tool, or a unified diff that applies but lands the line in the wrong place), except one 504 from the gateway after 90s on a trivial "count from 1 to 5" prompt — retried fine.

Live two-turn tool loop, through aimlapi.New(...) → CallWithTools, tools present on turn 1 and cleared on turn 2:

provider=aimlapi model=deepseek/deepseek-v4-flash
TURN1 ok: tool_calls=1 content=""
TURN1 call: id=call_00_zJQk8yixO2FpvIqsYEhA2155 name=get_open_ports args={"host": "10.0.0.7"}
TURN2 ok: content="The open ports on 10.0.0.7 are 22, 80, and 443."
usage turn2: {182 53 0 0 0 0}

The two-turn shape matters: this gateway returns 400 for "tools": null (and for null temperature, top_p, seed, tool_choice, response_format, stream, stream_options, max_tokens, max_completion_tokens), while accepting the same fields absent. A client that serialises a cleared field as an explicit null therefore succeeds on turn one of every agent loop and fails on turn two, with a mocked test suite staying green throughout. The Go path is safe — langchaingo declares all of these as pointers/slices with omitempty — and TestRequestOmitsUnsetSamplingParams and TestToolsAreOmittedNotNulledOnFollowUpTurn assert it against the real serialised bytes so it stays that way.

Not verified: the Settings UI and installer wizard changes were checked by type-checker, unit tests and the exhaustive-Record compile guard, not by clicking through a running stack — bringing up Postgres + the full compose stack was out of scope here. The goose migration was reviewed against 20260621_120000_add_minimax_provider.sql but not applied to a live database.

Security Considerations

No new dependencies, so CONTRIBUTING.md's license gate is unaffected. The API key is read from the environment like every other provider key and is added to GetSecretPatterns(), so it is redacted in logs the same way (the expectedCount assertion in config_test.go moved 30 → 31 accordingly). The attribution headers carry no user data — a fixed project name, repo URL, source slug and partner id — and are scoped to aimlapi.com, so a misconfigured AIMLAPI_SERVER_URL cannot leak them to a third party.

One upstream-facing note: GET https://api.aimlapi.com/v1/models returns 200 for any key, including a bogus one or none at all. Nothing in this PR validates a key that way, and nothing should.

Performance Impact

One extra RoundTripper in the chain for this provider only; it compares a hostname and, on a match, clones the request header. No effect on any other provider. Per-role latencies are in the report above.

Documentation Updates

• README.md updates
• Configuration documentation updates
• GraphQL schema updates
• Other: CLAUDE.md, backend/docs/config.md, backend/docs/database.md, .env.example, docker-compose.yml, .vscode/launch.json, examples/tests/aimlapi-report.md

Deployment Notes

New optional environment variables: AIMLAPI_API_KEY, AIMLAPI_SERVER_URL (default https://api.aimlapi.com/v1), AIMLAPI_PROVIDER. The provider is inert unless AIMLAPI_API_KEY is set. The goose migration 20260903_120000_add_aimlapi_provider.sql rebuilds the PROVIDER_TYPE enum to add aimlapi; its Down deletes rows using the new type before reverting, exactly as the MiniMax migration does.

Checklist

Code Quality

• My code follows the project's coding standards
• I have added/updated necessary documentation
• I have added tests to cover my changes
• All new and existing tests pass (pre-existing failures unchanged — see above)
• I have run go fmt and go vet (for Go code)
• I have run pnpm run lint (for TypeScript/JavaScript code)

Security

• I have considered security implications
• Changes maintain or improve the security model
• Sensitive information has been properly handled

Compatibility

• Changes are backward compatible
• Dependencies are properly updated (none added)

Documentation

• Documentation is clear and complete
• Comments are added for non-obvious code
• API changes are documented

Additional Notes

AIMLAPIToolCallIDTemplate is deliberately empty. The gateway multiplexes many vendors, so the tool-call id format follows the model actually serving the request, not the gateway; leaving it empty lets the shared detector derive it per model instead of asserting one format for the whole provider. (In practice deepseek/deepseek-v4-flash returned call_00_zJQk8yixO2FpvIqsYEhA2155, which matches the DeepSeek template — but that is a property of the upstream model, not of aimlapi.)

The provider icon in frontend/src/components/icons/aimlapi.tsx is a plain geometric "hub routing to several vendors" mark drawn in currentColor like the others, not a reproduction of any brand asset; drop in the real SVG when there is one.

No examples/configs/aimlapi.provider.yml is included: that file pattern exists for backends reached through the generic custom provider, and this one now ships its own embedded config.yml.